Information processing device, image acquisition system, information processing method, image information acquisition method, and program

ABSTRACT

[Solution] An information processing device according to the present invention includes: a representative luminance value specifying unit configured to, when luminance values constituting a plurality of fluorescence images of a measurement subject captured while a position of the measurement subject in a thickness direction is changed are sequentially rearranged from a highest luminance value on the basis of the fluorescence images for each of the fluorescence images corresponding to respective thickness positions, extract a luminance value ranked at a predetermined position from the highest luminance value and set the extracted luminance value as a representative luminance value of the fluorescence image at the thickness position to be noted; and a surface position specifying unit configured to use the representative luminance value for each of the fluorescence images and set the thickness position corresponding to the fluorescence image that gives the maximum representative luminance value as a position corresponding to a surface of the measurement subject.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase of International PatentApplication No. PCT/JP2015/077855 filed on Sep. 30, 2015, which claimspriority benefit of Japanese Patent Application No. JP 2014-248009 filedin the Japan Patent Office on Dec. 8, 2014. Each of the above-referencedapplications is hereby incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to an information processing device, animage acquisition system, an information processing method, an imageinformation acquisition method, and a program.

BACKGROUND ART

A laser confocal microscope disclosed below in Patent Literature 1, forexample, has been proposed as a laser confocal microscope (amicroendoscope) which uses an image guide fiber composed of a pluralityof optical fiber element wires. A microendoscope system proposed inPatent Literature 1, for example, transmits fluorescence generated byexciting an observation target with one photon through an image guidefiber and enables the generated fluorescence to be observed.

When an image captured by such a fluorescence observation system (i.e.,a fluorescence image) is observed, it is important to specify where aposition of a surface or a position of a focus of an observation targetis. Non-Patent Literature 1 described below discloses a method forspecifying a position of a surface or a position of a focus of anobservation target by applying an image recognition technology to afluorescence image obtained using a field-of-view (FOV) typefluorescence microscope.

CITATION LIST Patent Literature

-   Patent Literature 1: JP 2007-530197T

Non-Patent Literature

-   Non-Patent Literature 1: K. Kishima, “Simple way of pinpointing the    three-dimensional position of biomarkers in fluorescence microscopy    using a through-focus exposure method,” APPLIED OPTICS, 2011, Vol.    50, No. 25, p. 4989-   Non-Patent Literature 2: J. Rosen and G. Brooker, “Non-scanning    motionless fluorescence three-dimensional holographic microscopy,”    Nat. Photonics 2, 190-195 (2008)-   Non-Patent Literature 3: C. Maurer, S. Khan, S. Fassl, S. Bernet,    and M. Ritsch-Marte, “Depth of field multiplexing in microscopy,”    Opt. Express 18, 3023-3033 (2010)-   Non-Patent Literature 4: P. A. Dalgarno, H. I. C. Dalgarno, A.    Putoud, R. Lambert, L. Paterson, D. C. Logan, D. P. Towers, R. J.    Warburton, and A. H. Greenaway, “Multiplane imaging and three    dimensional nanoscale particle tracking in biological microscopy,”    Opt. Express 18, 877-883 (2010)

DISCLOSURE OF INVENTION Technical Problem

Here, the method disclosed in Non-Patent Literature 1 is a technology ofapplying an image recognition process to a fluorescence image obtainedusing a specific fluorescence microscope. Thus, it is extremelydifficult to apply the method disclosed in the above-describedNon-Patent Literature 1 to an object having a bright spot with anunclear size in a fluorescence image, like in a multi-photonfluorescence image obtained using a multi-photon fluorescencemicroscope.

Thus, a method in which a position of a surface of a measurement subjectcan be more simply specified regardless of an excitation process togenerate fluorescence or a type of a fluorescence microscope used influorescence measurement has been desired.

Therefore, the present disclosure takes the above circumstances intoconsideration and proposes an information processing device, an imageacquisition system, an information processing method, an imageinformation acquisition method, and a program which enable a position ofa surface of a measurement subject to be more simply specified withrespect to an arbitrary fluorescence image.

Solution to Problem

According to the present disclosure, there is provided an informationprocessing device including: a representative luminance value specifyingunit configured to, when luminance values constituting a plurality offluorescence images of a measurement subject captured while a positionof the measurement subject in a thickness direction is changed aresequentially rearranged from a highest luminance value on the basis ofthe fluorescence images for each of the fluorescence imagescorresponding to respective thickness positions, extract a luminancevalue ranked at a predetermined position from the highest luminancevalue and set the extracted luminance value as a representativeluminance value of the fluorescence image at the thickness position tobe noted; and a surface position specifying unit configured to use therepresentative luminance value for each of the fluorescence images andset the thickness position corresponding to the fluorescence image thatgives the maximum representative luminance value as a positioncorresponding to a surface of the measurement subject.

Further, according to the present disclosure, there is provided an imageacquisition system including: an imaging unit configured to generate aplurality of pieces of image data for fluorescence generated from ameasurement subject by radiating excitation light toward the measurementsubject and imaging the fluorescence of the measurement subject whilechanging a position of the measurement subject in a thickness direction;and an arithmetic processing unit configured to generate a plurality offluorescence images corresponding to respective thickness positions bycontrolling the imaging unit and performing data processing on each ofthe plurality of pieces of image data generated by the imaging unit. Theimaging unit includes a light source optical system configured to guideexcitation light for exciting the measurement subject with two or morephotons to generate fluorescence toward the measurement subject, animage guide fiber which is formed by bundling a plurality of multimodeoptical fiber element wires and is configured to transmit the excitationlight incident on one end to the measurement subject from the lightsource optical system and transmit an image of the measurement subjectformed on the other end using the fluorescence generated from themeasurement subject to the one end, and an imaging optical systemconfigured to scan the image of the measurement subject transmitted tothe one end of the image guide fiber at a scanning pitch that isnarrower than a size of a core of each of the plurality of optical fiberelement wires to perform imaging such that at least a part of an opticalfiber element wire-corresponding area which corresponds to each of theoptical fiber element wires is included in a plurality of images, andgenerate a plurality of pieces of image data of the measurement subject.The arithmetic processing unit includes a selection unit configured toselect, for each of a plurality of pixels constituting the optical fiberelement wire-corresponding area, a pixel value that has maximumluminance among the plurality of pieces of image data as arepresentative pixel value of the pixel, a captured imagere-constructing unit configured to re-construct the captured image ofthe measurement subject using the selected representative pixel valueand generate the fluorescence image, a representative luminance valuespecifying unit configured to, when luminance values constituting theplurality of fluorescence images captured while the position of themeasurement subject in the thickness direction is changed aresequentially rearranged from a highest luminance value on the basis ofthe fluorescence images for each of the fluorescence imagescorresponding to respective thickness positions, extract a luminancevalue ranked at a predetermined position from the highest luminancevalue and set the extracted luminance value as a representativeluminance value of the fluorescence image at the thickness position tobe noted, and a surface position specifying unit configured to use therepresentative luminance value for each of the fluorescence images andset the thickness position corresponding to the fluorescence image thatgives the maximum representative luminance value as a positioncorresponding to a surface of the measurement subject.

Further, according to the present disclosure, there is provided aninformation processing method including: extracting, when luminancevalues constituting a plurality of fluorescence images of a measurementsubject captured while a position of the measurement subject in athickness direction is changed are sequentially rearranged from ahighest luminance value on the basis of the fluorescence images for eachof the fluorescence images corresponding to respective thicknesspositions, a luminance value ranked at a predetermined position from thehighest luminance value and setting the extracted luminance value as arepresentative luminance value of the fluorescence image to be noted;and using the representative luminance value for each of thefluorescence images and setting the thickness position corresponding tothe fluorescence image that gives the maximum representative luminancevalue as a position corresponding to a surface of the measurementsubject.

Further, according to the present disclosure, there is provided an imageinformation acquisition method including: guiding excitation light forexciting a measurement subject with two or more photons to generatefluorescence toward the measurement subject; transmitting the excitationlight incident on one end of an image guide fiber which is formed bybundling a plurality of multimode optical fiber element wires toward themeasurement subject using the image guide fiber, and transmitting animage of the measurement subject formed on the other end using thefluorescence generated from the measurement subject to the one end whilechanging a position of the measurement subject in a thickness direction;scanning the image of the measurement subject transmitted to the one endof the image guide fiber at a scanning pitch that is narrower than asize of a core of each of the plurality of optical fiber element wiresto perform imaging such that at least a part of an optical fiber elementwire-corresponding area which corresponds to each of the optical fiberelement wires is included in a plurality of images, and generating aplurality of pieces of image data of the measurement subject; selecting,for each of a plurality of pixels constituting the optical fiber elementwire-corresponding area, a pixel value that has maximum luminance amongthe plurality of pieces of image data as a representative pixel value ofthe pixel; re-constructing the captured image of the measurement subjectusing the selected representative pixel value and generating thefluorescence image; extracting, when luminance values constituting aplurality of fluorescence images captured while the position of themeasurement subject in the thickness direction is changed aresequentially rearranged from a highest luminance value on the basis ofthe fluorescence images for each of the fluorescence imagescorresponding to respective thickness positions, a luminance valueranked at a predetermined position from the highest luminance value andsetting the extracted luminance value as a representative luminancevalue of the fluorescence image at the thickness position to be noted;and using the representative luminance value for each of thefluorescence images and setting the thickness position corresponding tothe fluorescence image that gives the maximum representative luminancevalue as a position corresponding to a surface of the measurementsubject.

Further, according to the present disclosure, there is provided aprogram causing a computer to realize: a representative luminance valuespecifying function of extracting, when luminance values constituting aplurality of fluorescence images of a measurement subject captured whilea position of the measurement subject in a thickness direction ischanged are sequentially rearranged from a highest luminance value onthe basis of the fluorescence images for each of the fluorescence imagescorresponding to thickness positions, a luminance value ranked at apredetermined position from the highest luminance value and setting theextracted luminance value as a representative luminance value of thefluorescence image to be noted; and a surface position specifyingfunction of using the representative luminance value for each of thefluorescence images and setting the thickness position corresponding tothe fluorescence image that gives the maximum representative luminancevalue as a position corresponding to a surface of the measurementsubject.

According to the present disclosure, using a plurality of fluorescenceimages captured while a position of a measurement subject in a thicknessdirection is changed, representative luminance values of thefluorescence images corresponding to thickness positions are specified,and a thickness position corresponding to a fluorescence image thatgives a maximum representative luminance value is set as a positioncorresponding to a surface of the measurement subject.

Advantageous Effects of Invention

According to the above-described present disclosure, a position of asurface of a measurement subject can be more simply specified withrespect to an arbitrary fluorescence image.

Note that the effects described above are not necessarily limitative.With or in the place of the above effects, there may be achieved any oneof the effects described in this specification or other effects that maybe grasped from this specification.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an illustrative diagram for describing fluorescence images.

FIG. 2 is a graph diagram for describing the fluorescence images.

FIG. 3 is a block diagram showing an example of a configuration of aninformation processing device according to a first embodiment of thepresent disclosure.

FIG. 4 is a block diagram showing an example of a configuration of animage information computation unit according to the embodiment.

FIG. 5 is an illustrative diagram for describing an image informationcomputation process according to the embodiment.

FIG. 6 is an illustrative diagram for describing an image informationcomputation process according to the embodiment.

FIG. 7 is an illustrative diagram for describing an image informationcomputation process according to the embodiment.

FIG. 8 is an illustrative diagram for describing an image informationcomputation process according to the embodiment.

FIG. 9 is an illustrative diagram for describing an image informationcomputation process according to the embodiment.

FIG. 10 is a flowchart showing an example of a flow of an informationprocessing method according to the embodiment.

FIG. 11 is a block diagram showing an example of a configuration of animage acquisition system according to a second embodiment of the presentdisclosure.

FIG. 12 is an illustrative diagram schematically showing an example of aconfiguration of an imaging unit according to the embodiment.

FIG. 13A is an illustrative diagram schematically showing an example ofa light source provided in the imaging unit according to the embodiment.

FIG. 13B is an illustrative diagram schematically showing an example ofa light source provided in the imaging unit according to the embodiment.

FIG. 13C is an illustrative diagram schematically showing an example ofa light source provided in the imaging unit according to the embodiment.

FIG. 13D is an illustrative diagram schematically showing an example ofa light source provided in the imaging unit according to the embodiment.

FIG. 14 is an illustrative diagram schematically showing a structure ofan image guide fiber provided in the imaging unit according to theembodiment.

FIG. 15A is an illustrative diagram for describing mode excitation whenlight converges on an end face of a multimode optical waveguide.

FIG. 15B is an illustrative diagram for describing mode excitation whenlight converges on an end face of a multimode optical waveguide.

FIG. 15C is an illustrative diagram for describing mode excitation whenlight converges on an end face of a multimode optical waveguide.

FIG. 15D is an illustrative diagram for describing mode excitation whenlight converges on an end face of a multimode optical waveguide.

FIG. 16A is an illustrative diagram schematically showing a scanningmethod of the image guide fiber of the imaging unit according to theembodiment.

FIG. 16B is an illustrative diagram schematically showing a scanningmethod of the image guide fiber of the imaging unit according to theembodiment.

FIG. 17 is an illustrative diagram schematically showing a specificexample of the imaging unit according to the embodiment.

FIG. 18A is an illustrative diagram for describing an end unit of theimage guide fiber according to the embodiment.

FIG. 18B is an illustrative diagram for describing an end unit of theimage guide fiber according to the embodiment.

FIG. 19 is a block diagram showing an example of a configuration of anarithmetic processing unit according to the embodiment.

FIG. 20 is a block diagram showing an example of a configuration of adata processing unit according to the embodiment.

FIG. 21 is an illustrative diagram for describing a representative pixelvalue selection process of the data processing unit according to theembodiment.

FIG. 22 is an illustrative diagram for describing a representative pixelvalue selection process of the data processing unit according to theembodiment.

FIG. 23 is an illustrative diagram for describing a representative pixelvalue selection process of the data processing unit according to theembodiment.

FIG. 24 is a block diagram showing an example of a hardwareconfiguration of an information processing device and an arithmeticoperation processing unit according to an embodiment of the presentdisclosure.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, (a) preferred embodiment(s) of the present disclosure willbe described in detail with reference to the appended drawings. In thisspecification and the appended drawings, structural elements that havesubstantially the same function and structure are denoted with the samereference numerals, and repeated explanation of these structuralelements is omitted.

Note that description will be provided in the following order.

1. Regarding fluorescence image

2. First embodiment (Information processing device)

3. Second embodiment (Image acquisition system)

(Regarding Fluorescence Image)

Before an information processing device and an image acquisition systemaccording to an embodiment of the present disclosure are described, afluorescence image to be noted in the present disclosure will be brieflydescribed.

As endoscope technologies have developed in recent years, surgicaloperations or intraoperative diagnoses have been conducted underendoscopes. Here, when it is determined whether a cell to be noted is anormal cell or a cancer cell, a predetermined fluorescent dye isinjected into an organ to be noted, and then a fluorescence image of theorgan is observed using a fluorescence microscope connected to anendoscope.

Here, when a specimen (a measurement subject) having a three-dimensionalstructure is observed using a fluorescence microscope, as withobservation of a three-dimensionally cultured cell as well as theabove-described in-vivo somatoscopy, it is more difficult to determinewhich position is a surface in comparison to a case of bright fieldobservation.

It is not obvious in a fluorescence image which bright spot observed inthe image is present at a position close to a surface (a position closeto an object lens). For this reason, a user compares positions at whichbright spots have a high contrast or positions at which contours ofbright spots are apparent to each other for each bright spot, selects abright spot focused at a position closest to the object lens among thebright spots, and then specifies a position of a surface of a cell.

When a measurement subject is fluorescent beads having a uniformdiameter that is uniform and smaller than a focal depth, anyone canspecify a position of a surface even though he or she is not a skilledperson. However, when a substance having a variable shape and a varietyof thicknesses, like cells or the like, is a measurement subject, it isnot possible to specify a position of a surface thereof unless athree-dimensional structure of the measurement subject can be imagined,and thus specification of a position of a surface of a measurementsubject depends on user proficiency. For this reason, a method in whicha position of a surface of a measurement subject can be simply specifiedin a fluorescence image without relying on user proficiency has beendesired.

Applying the image recognition technology as described in Non-PatentLiterature 1 is considered in order to, for example, specify a positionof a surface of a measurement subject in a fluorescence image. However,as described above, the technology of Non-Patent Literature 1 is atechnology applicable to a fluorescence image obtained using a specificfluorescence microscope, and using the technology entails difficulty. Inaddition, use of an auto-focus technology for detecting a position ofslide glass is also considered, however, the technology is not appliedunder an environment in which no slide glass is used as in in-vivosomatoscopy.

Therefore, the inventor focused on using some kind of information thatcan be obtained from a fluorescence image to specify a position of asurface regardless of the use of a fluorescence microscope or afluorescence excitement process. One kind of information considered asthe information is an average luminance value of fluorescence images.

FIG. 1 is an illustrative diagram for describing fluorescence images,and FIG. 2 is a graph diagram for describing the fluorescence images.

With respect to the fluorescence images, brightness of bright spotsgenerally brightens as the fluorescence images are observed on a sidecloser to a surface, and brightness of the bright spots darkens as thefluorescence images are observed on a side farther from the surfacesince a thickness that the fluorescence should permeate is increased, asschematically shown in FIG. 1. In addition, fluorescence images tend tobe bright overall at observation positions at which the fluorescenceimages are out of focus. In addition, the number of bright spots differsdepending on observation positions of a measurement subject in athickness direction (a depth direction) as schematically shown in FIG.1.

FIG. 2 is a diagram in which a change in average luminance values of thefluorescence images when fluorescence that is generated by a certaincell is observed from a deep part side of the cell to a surface sidewith an interval of 5 μm is plotted. In FIG. 2, the horizontal axisrepresents numbers (image numbers) associated with the fluorescenceimages sequentially captured from the deep part side, and the verticalaxis represents the average luminance values.

As can be seen from FIG. 2, when an average luminance value of afluorescence image to be noted is employed as one kind of informationobtained from luminance values of the fluorescence images, the averageluminance value increases toward the surface side of a measurementsubject. For this reason, it is not possible to specify a position of asurface of the measurement subject when attention is paid to the averageluminance values.

As a result of further intense examination on the basis of theabove-described knowledge, the present inventor has decided to focus ona distribution of luminance values itself, rather than an averageluminance value of fluorescence images, and has completed an informationprocessing device and an image acquisition system according to anembodiment of the present disclosure to be described in detail below.

First Embodiment

An information processing device according to a first embodiment of thepresent disclosure will be described in detail below with reference toFIGS. 3 to 9. FIG. 3 is a block diagram showing an example of aconfiguration of the information processing device according to thepresent embodiment, and FIG. 4 is a block diagram showing an example ofa configuration of an image information computation unit provided in theinformation processing device according to the present embodiment. FIGS.5 to 9 are illustrative diagrams for describing an image informationcomputation process according to the present embodiment.

<Overall Configuration of Information Processing Device>

First, an overall configuration of an information processing device 10according to the present embodiment will be described with reference toFIG. 3.

The information processing device 10 according to the present embodimentis a device which acquires a fluorescence image of a measurement subjectcaptured by an imaging unit 20, uses the fluorescence image, andcomputes image information relevant to the fluorescence image includingat least a position of a surface of the measurement subject.

The information processing device 10 may be an information processingdevice such as any of a variety of computers and servers providedoutside the imaging unit 20, or may be an arithmetic operation chipconstituted by a central processing unit (CPU), a read only memory(ROM), a random access memory (RAM), and the like installed in theimaging unit 20.

Here, a measurement subject is not particularly limited as long as it isa substance that emits fluorescence, and the measurement subject may bean inanimate object such as a fluorescence bead or an animate objectsuch as any of a variety of kinds of cells. In addition, fluorescenceemitted from a measurement subject may be emitted by the measurementsubject itself, or may be emitted by any of a variety of fluorescentdyes added to the measurement subject. Furthermore, there is noparticular limit on an excitation process of such fluorescence, and itmay be fluorescence emitted from a fluorescent substance excited in aone-photon process, or may be fluorescence emitted from a fluorescencesubstance excited in a multiple-photon processes, for example, atwo-photon process.

In addition, the imaging unit 20 that captures such a fluorescence imageis a unit which radiates excitation light having a predeterminedwavelength toward a measurement subject, detects fluorescence generatedfrom the measurement subject, and thereby generates image data regardingthe generated fluorescence. Any device such as one of a variety of kindsof fluorescence microscopes can be used as the imaging unit 20 as longas the device can capture fluorescence images while changing a positionof a measurement subject in the thickness direction.

The information processing device 10 according to the present embodimentmainly has a data acquisition unit 101, an image information computationunit 103, a display control unit 105, and a storage unit 107 asillustrated in FIG. 3.

The data acquisition unit 101 is realized by, for example, a CPU, a ROM,a RAM, a communication device, and the like. The data acquisition unit101 acquires image data of a plurality of fluorescence images generatedby the imaging unit 20 in which the measurement subject has a differentthickness direction from the imaging unit 20. The image data of theplurality of fluorescence images acquired by the data acquisition unit101 is transmitted to the image information computation unit 103 to bedescribed below. In addition, the data acquisition unit 101 mayassociate the acquired image data of the plurality of fluorescenceimages with time information regarding a date, time, and the like onwhich the image data is acquired and store the associated data in thestorage unit 107 to be described below as history information.

The image information computation unit 103 is realized by, for example,a CPU, a ROM, a RAM, and the like. The image information computationunit 103 computes image information including at least informationregarding a position corresponding to a surface of the measurementsubject using the plurality of fluorescence images transmitted from thedata acquisition unit 101, in which the measurement subject has adifferent thickness direction. In addition, the image informationcomputation unit 103 may also compute information regarding a scatteringcoefficient of the measurement subject as image information. When theimage information mentioned above has been computed, the imageinformation computation unit 103 outputs information regarding thecomputed image information to the display control unit 105. Accordingly,the image information regarding the fluorescence images of themeasurement subject S is output to a display unit (not illustrated)provided in the information processing device 10 or any of variouscomputers that can communicate with the information processing device10. In addition, the image information computation unit 103 may outputthe obtained image information to any of a variety of recording media,computers, or the like, or to a paper medium using an output device suchas a printer. In addition, the image information computation unit 103may associate the image information regarding the fluorescence images ofthe measurement subject S with time information regarding a date, time,and the like at which the information was computed and store theassociated data in the storage unit 107 or the like as historyinformation.

Note that a detailed configuration of the image information computationunit 103 will be described below.

The display control unit 105 is realized by, for example, a CPU, a ROM,a RAM, an output device, and the like. The display control unit 105performs display control to display a position of a surface of themeasurement subject S or various processing results including the imageinformation regarding the scattering coefficient of the measurementsubject S or the like transmitted from the image information computationunit 103 on an output device such as a display of the informationprocessing device 10 and an output device provided outside of theinformation processing device 10. Accordingly, a user of the informationprocessing device 10 can immediately ascertain the various processingresults with regard to the measurement subject S.

The storage unit 107 is realized by, for example, a RAM of theinformation processing device 10 according to the present embodiment, astorage device, or the like. The storage unit 107 appropriately recordsvarious parameters or process developments that need to be saved by theinformation processing device 10 according to the present embodiment toperform any process, or various databases, programs, and the like. Thestorage unit 107 enables the data acquisition unit 101, the imageinformation computation unit 103, the display control unit 105, and thelike to freely perform data read and write processes.

[Regarding Configuration of Image Information Computation Unit 103]

Next, a detailed configuration of the image information computation unit103 of the information processing device 10 according to the presentembodiment will be described with reference to FIGS. 4 to 9.

The image information computation unit 103 according to the presentembodiment has, for example, a representative luminance value specifyingunit 111, a surface position specifying unit 113, a scatteringcoefficient computation unit 115, and a result output unit 117 asillustrated in FIG. 4.

The representative luminance value specifying unit 111 is realized by,for example, a CPU, a ROM, a RAM, and the like. The representativeluminance value specifying unit 111 executes, on the basis of theplurality of fluorescence images that were captured d while the positionof the measurement subject S is changed in the thickness direction, aprocess (a sorting process) of sequentially rearranging luminance valuesconstituting the plurality of fluorescence images from the highestluminance value for each of the fluorescence images corresponding tothickness positions thereof. Thereafter, the representative luminancevalue specifying unit 111 extracts a luminance value ranked at apredetermined position from the highest luminance value and sets theextracted luminance value as a representative luminance value of afluorescence image at a thickness position to be noted.

FIG. 5 shows results of a sorting process for luminance values performedusing fluorescence images obtained by imaging fluorescence from eGFPgenerated from the measurement subject S (specifically, an eGFP-RFPexpression cancer tissue cell created in an individual mouse: MKN 45 (ahuman gastric cancer epithelial cell)) while changing fluorescence by 5μm in a depth direction. In FIG. 5, the horizontal axis representsnumbers (image numbers: #1 to #60) linked to fluorescence images thatare sequentially captured from the depth side, and a higher numberdenotes that the florescence image is approaching a surface side. Inaddition, the vertical axis represents luminance values.

If sequential sorting of luminance values of all pixels constituting onefluorescence image from the highest luminance value to the lowestluminance value is performed for each of the plurality of fluorescenceimages and then a change in the highest luminance values of thefluorescence images is plotted, a curve denoted as “Maximum” in FIG. 5is obtained. Likewise, if luminance values ranked in the top 1% of thehighest luminance values (for example, the 10,000^(th) luminance valuefrom the top in case of data of a fluorescence image having one millionpixels) are plotted, a curve denoted as “99%” in FIG. 5 is obtained.Likewise for other values in FIG. 5, changes of luminance values rankedin the top 5% (“95%” in the diagram), the top 10% (“90%” in thediagram), the above-described 20% (“80%” in the diagram), 30% (“70%” inthe diagram), 40% (“60%” in the diagram), 50% (“50%” in the diagram),90% (“10%” in the diagram), and a minimum pixel value (“minimum” in thediagram) are plotted.

Here, for the image information computation unit 103 according to thepresent embodiment, a “fluorescence image that is the brightest and infocus is considered to be a fluorescence image for a surface of themeasurement subject S. Here, if the fluorescence images have asubstantially equal probability of having bright spots, the same brightspots can be regarded as being observed when luminance values of allpixels of the fluorescence images are sequentially sorted from thehighest luminance value. As illustrated in FIG. 5, if a curveexponentially changes when a change in the luminance values inaccordance with a change in a thickness position is plotted, theabove-described precondition can be determined to be substantiallysatisfied. Thus, if a plot of ranked positions at which luminance valuesexponentially change is specified by plotting the change in theluminance values at specific ranked positions of the fluorescence imageshaving different depth directions and a position of a highest luminancevalue can be specified on the basis of the plot as illustrated in FIG.5, a position of a surface of the measurement subject S can be decided.

Referring to the state of the luminance values corresponding to“Maximum” in FIG. 5, the change in the luminance values is not even, butthere are several peaks in the number #1 to the number #30. The reasonfor this is considered to be that electric noise caused by a laser, animage sensor, or the like that is used to generate the fluorescenceimages has been superimposed. In addition, luminance values aresaturated from the number #35.

In addition, referring to the state of the change in the luminancevalues ranked in the top 1% (the plot of “99%”) in FIG. 5, asubstantially exponential change in luminance values is found from thenumber #1 to the number #40. In addition, referring to the change in theluminance values ranked in the top 5% (the plot of (95%)), while an evenchange is found from the number #1 to the number #40, no exponentialchange in luminance values is found, unlike the plot of “99%.” Inaddition, the change in the luminance values ranked in the top 10% orhigher monotonously increases from the number #1 to the number #60, andthus the luminance values are not used to specify a position of asurface.

Thus, the representative luminance value specifying unit 111 can specifythe position of the surface of the measurement subject S to be noted bydealing with the luminance values ranked in the top 1% (i.e., theluminance values on the plot of “99%” in FIG. 5) among the plots shownin FIG. 5 as representative luminance values.

Here, a rank from the top position at which luminance values to be notedare positioned may be appropriately selected in accordance with adensity of bright spots of a fluorescence image to be noted, however,for example, a rank in the range of the top 0.5% to 5% of the number ofall pixels constituting one fluorescence image with reference to ahighest luminance value is preferable. A case of a rank in the top 0.5%or less is not preferable because there is a possibility of variouskinds of electric noise being superimposed on the luminance values likethe plot of “Maximum” in FIG. 5. In addition, a case of a rank exceedingthe top 5% is not preferable because there is a possibility of luminancevalues increasing monotonously, like the plot of “90%” in FIG. 5.

The representative luminance value specifying unit 111 extracts aluminance value ranked at a predetermined position from the highestluminance value of each of fluorescence images as a representativeluminance value.

The representative luminance value specifying unit 111 outputsinformation of the representative luminance value extracted as describedabove to the surface position specifying unit 113 and the scatteringcoefficient computation unit 115 to be described below.

The surface position specifying unit 113 is realized by, for example, aCPU, a ROM, a RAM, and the like. The surface position specifying unit113 specifies a position corresponding to a surface of the measurementsubject S on the basis of a representative luminance value of eachfluorescence image extracted by the representative luminance valuespecifying unit 111. Specifically, the surface position specifying unit113 sets a thickness position corresponding to a fluorescence image thatgives a maximum representative luminance value among the representativeluminance values of the respective fluorescence images as a positioncorresponding to the surface of the measurement subject S to be noted.

In the example shown in FIG. 5, for example, the plot of the luminancevalues denoted as “99%” in the diagram is set as a plot expressing astate of a change in representative luminance values, however, thesurface position specifying unit 113 specifies a position of the number#43 (i.e., a position 5 μm×43=215 μm upward from a measurement startposition), which is a position at which the maximum luminance value isgiven among the plots expressing the changes of the representativeluminance values, as the position of the surface of the measurementsubject S to be noted.

The surface position specifying unit 113 outputs information regardingthe position of the surface of the measurement subject S specified asdescribed above to the scattering coefficient computation unit 115 andthe result output unit 117 to be described below.

The scattering coefficient computation unit 115 is realized by, forexample, a CPU, a ROM, a RAM, and the like. The scattering coefficientcomputation unit 115 uses the representative luminance values of therespective fluorescence images and computes a scattering coefficient ofthe measurement subject from a degree of change in the representativeluminance values in the thickness direction.

As described above, luminance values at the ranks extracted as therepresentative luminance values exponentially change in accordance witha change in the position in the thickness direction. Thus, by focusingon the change in the representative luminance values in the thicknessdirection, a scattering coefficient of the measurement subject S can beobtained from the degree of change.

In more detail, the scattering coefficient computation unit 115 computesa scattering coefficient using representative luminance values at threethickness positions having equal intervals on the basis ofrepresentative luminance values of fluorescence images corresponding todeeper parts than the position corresponding to the surface of themeasurement subject S. A scattering coefficient computation process bythe scattering coefficient computation unit 115 will be described indetail below with reference to FIG. 6.

It is assumed that a fluorescence image is obtained by capturingfluorescence generated when the measurement subject S is excited with N(N is an integer greater than or equal to 1) photons, and asschematically illustrated in FIG. 6, a representative luminance value ata thickness position x_(i) (i=1, 2, or 3) is denoted by A_(i) (i=1, 2,or 3), an interval between two adjacent thickness positions is denotedby dx, and a scattering coefficient of the measurement subject S isdenoted by R_(S). If a background luminance value is set to BG for theplot of the representative luminance values illustrated in FIG. 5, thetwo following formulas 101 and 103 are established from a definition ofthe scattering coefficient R_(S).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 1} \right\rbrack & \; \\{\frac{A_{1} - {BG}}{A_{2} - {BG}} = {\exp\left( {{- N} \cdot R_{s} \cdot {dx}} \right)}} & \left( {{Formula}\mspace{14mu} 101} \right) \\{\frac{A_{2} - {BG}}{A_{3} - {BG}} = {\exp\left( {{- N} \cdot R_{s} \cdot {dx}} \right)}} & \left( {{Formula}\mspace{14mu} 103} \right)\end{matrix}$

Here, if the background BG is simultaneously erased in the formulas 101and 103, the following formula 105 can be obtained. Here, a unit of thescattering coefficient R_(S) is [1/mm] in the following formula 105.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 2} \right\rbrack & \; \\{R_{s} = {{- \frac{1}{N \cdot {dx}}} \cdot {\ln\left( \frac{A_{1} - A_{2}}{A_{2} - A_{3}} \right)}}} & \left( {{Formula}\mspace{14mu} 105} \right)\end{matrix}$

Here, in the formula 105, N is a known parameter decided in accordancewith a fluorescence microscope used when the fluorescence images arecaptured, dx is a known parameter set when the three representativeluminance values are extracted, and A₁ to A₃ are the representativeluminance values that can be obtained from the fluorescence images.Thus, the scattering coefficient R_(S) of the measurement subject S canbe computed from the representative luminance values using the formula105.

FIGS. 7 to 9 show scattering coefficients R_(S) of the measurementsubject S computed with the formula 105 using fluorescence imagescaptured using a two-photon fluorescence microscope. Note that thefluorescence images are the same as those used in generating FIG. 5.Here, FIG. 7 shows computation results of the scattering coefficientsR_(S) when dx is set to 10 μm, FIG. 8 shows computation results of thescattering coefficients R_(S) when dx is set to 25 μm, and FIG. 9 showscomputation results of the scattering coefficients R_(S) when dx is setto 35 μm.

Here, since the position corresponding to the surface is the number #43as is obvious from FIG. 5, data from the number #43 does not include themeasurement subject S and thus it is meaningless for the data to be usedto measure the scattering coefficient R_(S). In addition, with regard tothe representative luminance values, a change in luminance valuesprogresses exponentially, but it is better to note a position at whichthe change is expressed as exponentially as possible (i.e., at aposition at which the luminance values sharply change) among positionsof the representative luminance values. When the plot of “99%” of FIG. 6is focused on, it can be ascertained that luminance values sharplychange at positions of the image numbers #20 to #40. Thus, thescattering coefficient computation unit 115 sets a least uneven value atthe position at which the luminance values sharply change among thescattering coefficients R_(S) computed on the basis of the formula 105as the scattering coefficient R_(S) of the measurement subject S to benoted.

The result is about 5 (1/mm) at all of the positions of the numbers #20to #40 among the results shown in FIGS. 7 to 9. Thus, the scatteringcoefficient computation unit 115 computes the scattering coefficientR_(S) of the measurement subject S to be noted to be 5 (1/mm).

The scattering coefficient computation unit 115 outputs informationregarding the scattering coefficient R_(S) of the measurement subject Scomputed as described above to the result output unit 117 to bedescribed below.

The result output unit 117 is realized by, for example, a CPU, a ROM, aRAM, an output device, a communication device, and the like. The resultoutput unit 117 outputs information regarding the position of thesurface of the measurement subject S specified by the surface positionspecifying unit 113 and information regarding the scattering coefficientR_(S) of the measurement subject S computed by the scatteringcoefficient computation unit 115.

Examples of functions of the information processing device 10 accordingto the present embodiment have been described above. The respectiveconstituent elements may be configured using universal members andcircuits, or may be configured using hardware specialized for thefunctions of the constituent elements. In addition, all of the functionsof the constituent elements may be fulfilled by a CPU and the like.Thus, a configuration to be used can be appropriately changed inaccordance with a technical level of any occasion at which the presentembodiment is implemented.

Note that a computer program for realizing each function of theinformation processing device according to the above-described presentembodiment can be produced and installed in a personal computer and thelike. In addition, a computer-readable recording medium on which thecomputer program is stored can also be provided. The recording mediumis, for example, a magnetic disk, an optical disc, a magneto-opticaldisc, a flash memory, or the like. Furthermore, the computer program maybe distributed through, for example, a network, without using arecording medium.

<Regarding Flow of Information Processing Method>

Next, an example of a flow of an information processing method executedby the information processing device 10 according to the presentembodiment will be briefly described with reference to FIG. 10. FIG. 10is a flowchart showing the example of the flow of the informationprocessing method according to the present embodiment.

In the information processing device 10 according to the presentembodiment, first, the data acquisition unit 101 acquires a plurality offluorescence images obtained from different thickness positions of themeasurement subject S (Step S101). The data acquisition unit 101 outputsinformation regarding the plurality of acquired fluorescence images tothe representative luminance value specifying unit 111 of the imageinformation computation unit 103.

The representative luminance value specifying unit 111 extractsrepresentative luminance values from the plurality of respectivefluorescence images using the fluorescence images output from the dataacquisition unit 101 (Step S103). Then, the representative luminancevalue specifying unit 111 outputs information regarding the extractedrepresentative luminance values to the surface position specifying unit113 and the scattering coefficient computation unit 115.

The surface position specifying unit 113 specifies a position at which amaximum representative luminance value is given as a positioncorresponding to a surface of the measurement subject S on the basis ofthe extracted representative luminance values (Step S105). The surfaceposition specifying unit 113 outputs information regarding the specifiedposition of the surface of the measurement subject S to the scatteringcoefficient computation unit 115 and the result output unit 117.

In addition, the scattering coefficient computation unit 115 computesthe scattering coefficient R_(S) using the above-described formula 105on the basis of the extracted representative luminance values (StepS107). The scattering coefficient computation unit 115 outputsinformation regarding the computed scattering coefficient R_(S) to theresult output unit 117.

The result output unit 117 outputs the information regarding theposition of the surface specified by the surface position specifyingunit 113 and the information regarding the scattering coefficientcomputed by the scattering coefficient computation unit 115 to theoutside (Step S109). Accordingly, a user of the information processingdevice 10 can ascertain image information regarding the fluorescenceimages of the measurement subject S.

The example of the flow of the information processing method accordingto the present embodiment has been briefly described above withreference to FIG. 10.

According to the information processing device 10 and the informationprocessing method of the present embodiment as described above, asurface position specifying method that does not depend on visualobservation (in other words, that does not depend on a technique of auser) can be provided.

In addition, by directly analyzing a fluorescence image using theinformation processing device 10 and the information processing methodaccording to the present embodiment, the scattering coefficient R_(S) ofthe measurement subject S can be computed. If the scattering coefficientR_(S) of the measurement subject S is known, an intensity (power)necessary for acquiring a fluorescence image can be estimated, and thusimportant information for appropriately acquiring a fluorescence imagecan be obtained.

Furthermore, since a scattering coefficient has a possibility of havinga relation with a state of a cell such as fibrosis, there is apossibility of a measurement of a scattering coefficient serving as anindex for evaluating a function of an organ. Using the informationprocessing device 10 and the information processing method according tothe present embodiment, the scattering coefficient R_(S) can be computedif information can be acquired through observation from one side of themeasurement subject S and the measurement subject S has a thickness ofabout dozens micrometers. Thus, using the information processing device10 and the information processing method according to the presentembodiment, even information of an organ having high tissue porositysuch as a lung can be acquired, and information can be acquired from asmall tissue sample from an organ such as a liver.

With regard to a degree of a margin with which a cancer tissue is to beexcised in a surgery for a digestive organ cancer such as a livercancer, a doctor ascertains a degree of inflammation or induration ofthe organ during the surgery and excises a sufficient area of the cancertissue securing an extra margin if there is likely to be littleinduration, or leaves a large area of an organ tissue having a smallmargin if there is likely to be induration. When the surgery is alaparotomy, the doctor ascertains a degree of inflammation or indurationof an organ of a patient by touching the organ with his or her hand orvisually checking the organ. However, as endoscopic surgeries havedeveloped in recent years, minimally invasive endoscopic surgeries havebeen used most frequently. Here, there also is a method oftranscutaneously diagnosing hardness of a liver using ultrasonic waves(elastography), however, elastography is very expensive and thus usingthe technique in an operating room of a gastroenterological surgery isnot normal. Thus, a degree of inflammation or induration of a liver of apatient is normally ascertained only using a technique of determiningthe degree of inflammation or induration by color using a camera inendoscopic surgery. For this reason, accuracy of information on a liverfunction of a patient is lower than a recovery surgery performed in thepast, and thus a risk of excising a large margin even though indurationhas progressed or the like is heightened.

Here, it has been reported that cancerous organs have scatteringcoefficients or absorption coefficients that are about 15% lower thanthose of normal tissues, and since the information processing device 10and the information processing method according to the presentembodiment can compute a scattering coefficient of a tissue from afluorescence image, there is great merit in deciding a position to beexcised and an intraoperative diagnosis of a cell in agastroenterological surgery. In addition, the merit also contributes tothe discovery of cancer cells in respiratory internal medicine,respiratory surgery, and urology. As described above, the informationprocessing device 10 and the information processing method according tothe present embodiment also provide useful information in the field ofmedicine.

Second Embodiment

Next, as a second embodiment of the present disclosure, an imageacquisition system that has an imaging function of imaging afluorescence image of a measurement subject and the function ofcomputing image information including a position of a surface and ascattering intensity of a measurement subject described in the firstembodiment will be described in detail with reference to FIGS. 11 to 23.

As will be described in detail, in the image acquisition systemaccording to the present embodiment, fluorescence images are acquiredusing an image guide fiber constituted by a plurality of optical fiberelement wires. The present inventor has separately reviewed a method fordetecting fluorescence generated through a multi-photon excitationprocess in order to use such an image guide fiber. As a result, thepresent inventor has gained knowledge that, in order to increaseluminance of fluorescence through the multi-photon excitation process,it is preferable that a guided wave of excitation light using an imageguide fiber is in a single mode (more specifically, a zero-order mode)and, if an image guide fiber constituted by optical fiber element wiresin a multi-mode is used as it is, a decrease or unevenness in luminanceof the fluorescence occurs so as to make it impossible to acquire afluorescence image using a multi-photon excitation process.

Here, when fluorescence in a multi-photon excitation process is measuredusing an image guide fiber, optical fiber element wires constituting theimage guide fiber are also considered to be set as optical fiber elementwires in a single mode. In this case, in order to reduce crosstalkbetween adjacent optical fiber element wires when the optical fiberelement wires in the single mode are used, it is necessary to reduce adifference in refractive index between a core and a cladding of theoptical fiber. However, reducing the difference in refractive indexbetween the core and the cladding means an increase in diffusion ofelectric field intensity distribution toward the cladding. Thus, inorder to reduce crosstalk between adjacent optical fiber element wires,it is important to widen intervals between the adjacent optical fiberelement wires.

Resolution of the image guide fiber depends on arrangement intervals ofthe optical fiber element wires, and as the arrangement intervals of theoptical fiber element wires is narrowed, the resolution obtained isincreased. Thus, when the optical fiber element wires in the single modeare used and intervals of the optical fiber element wires are widened,while another mechanism that scans the optical fiber to obtain the samedegree of resolution as that of a normal image guide fiber is necessary,it is difficult to narrow a diameter of the optical fiber element wires.

Furthermore, reducing the difference in refractive index between thecore and the cladding of the optical fiber leads a numerical aperture(NA) of the optical fiber element wires being lowered, and thus in orderto acquire a signal of fluorescence in a multi-photon excitation processwith high efficiency, an effort of using a double-clad fiber or the likeis important.

On the basis of the above-described knowledge, the present inventor hasintensively investigated an image acquisition system which can acquire afluorescence image using stable multi-photon excitement even when animage guide fiber composed of optical fiber element wires in amulti-mode is used. As a result, the present inventor found that (1) ifa single mode is excited on an incident end face of an image guidefiber, there is a case in which fluorescence probably reaches asample-side end face in the single mode, and (2) data of excitationlight that has reached the sample-side end face in the single mode canbe acquired by acquiring images a plurality of times and selecting thehighest fluorescence value, and thus the present inventor has developedan image acquisition system according to the second embodiment of thepresent disclosure to be described below.

<Regarding Image Acquisition System>

[Overall Configuration of Image Acquisition System]

First, an overall configuration of an image acquisition system 30according to the second embodiment of the present disclosure completedon the basis of the above-described knowledge will be described withreference to FIG. 11. FIG. 11 is an illustrative diagram schematicallyshowing a configuration of the image acquisition system according to thepresent embodiment.

The image acquisition system 30 according to the present embodiment is asystem which radiates excitation light having a predetermined wavelengthtoward a measurement subject that is an observation target to generatefluorescence from the measurement subject in a multi-photon excitationprocess, acquires a captured image of the imaging subject based on thefluorescence, and computes image information as described in the firstembodiment on the basis of the obtained captured image. The imageacquisition system 30 has an imaging unit 40 and an arithmeticprocessing unit 50 as illustrated in FIG. 11.

The imaging unit 40 is a unit which radiates the excitation light havinga predetermined wavelength toward the measurement subject, detectsfluorescence generated through a multi-photon excitation process, andthereby generates image data regarding the generated fluorescence. Theimage data generated by the imaging unit 40 is output to the arithmeticprocessing unit 50. Details of a configuration of the imaging unit 40will be described below.

The arithmetic processing unit 50 is a unit which comprehensivelycontrols imaging processing on the measurement subject that is performedby the imaging unit 40, performs an arithmetic process to be describedbelow on the image data generated by the imaging unit 40, generates acaptured image of the measurement subject, and computes imageinformation thereof.

The arithmetic processing unit 50 may be an information processingdevice such as any of a variety of computers or servers provided outsidethe imaging unit 40, or may be an arithmetic chip that is installed inthe imaging unit 40 and composed of a CPU, a ROM, a RAM, and the like.

Details of a configuration of the arithmetic processing unit 50 will bedescribed below.

[Configuration of Imaging Unit 40]

Next, details of a configuration of the imaging unit 40 according to thepresent embodiment will be described with reference to FIGS. 12 to 18B.

FIG. 12 is an illustrative diagram schematically showing an example ofthe configuration of the imaging unit according to the presentembodiment. FIGS. 13A to 13D are illustrative diagrams schematicallyshowing examples of a light source provided in the imaging unitaccording to the present embodiment. FIG. 14 is an illustrative diagramschematically showing a structure of an image guide fiber provided inthe imaging unit according to the present embodiment. FIGS. 15A to 15Dare illustrative diagrams for describing mode excitation when lightconverges on an end face of a multi-mode optical waveguide. FIGS. 16Aand 16B are illustrative diagrams schematically showing scanning methodsof the image guide fiber of the imaging unit according to the presentembodiment. FIG. 17 is an illustrative diagram schematically showing aspecific example of the imaging unit according to the presentembodiment. FIGS. 18A and 18B are illustrative diagrams for describingan end unit of the image guide fiber according to the presentembodiment.

The imaging unit 40 of the image acquisition system 30 according to thepresent embodiment mainly has a light source optical system 401, animage guide fiber 403, and an imaging optical system 405 as illustratedin FIG. 12.

The light source optical system 401 is an optical system which guidesexcitation light having a predetermined wavelength for generatingfluorescence by exciting the measurement subject S with two or morephotons (i.e., excitation using multiple photons) toward the measurementsubject S. The light source optical system 401 is constituted by a laserlight source which emits the excitation light having a predeterminedwavelength and optical elements such as various lenses, various mirrors,and various filters that guide the excitation light emitted from thelight source toward the measurement subject S.

A detailed disposition of the various optical elements in the lightsource optical system 401 is not particularly limited, and a dispositionof a known optical system can be employed.

In addition, a laser light source of the light source optical system 401is also not particularly limited, and any of a variety of light sourcessuch as various semiconductor lasers, solid-state lasers, and gas laserscan be used. By using a light source which uses any of a variety ofsemiconductor light sources as the laser light source, the imageacquisition system 30 can be further miniaturized.

A semiconductor laser and a light source which uses a wavelengthconversion unit which converts a wavelength of light of thesemiconductor laser as illustrated in FIG. 13A to 13D can be used, forexample, as a light source which uses a semiconductor laser that can beused as the laser light source unit of the light source optical system401.

FIG. 13A schematically illustrates a master oscillator 411, which isconstituted by a semiconductor laser and a resonator, as an example of asemiconductor laser that can be used in the laser light source unit. Themaster oscillator 411 provided as the laser light source unit isconstituted by a semiconductor laser unit 412 that can emit laser lighthaving a predetermined wavelength (e.g., a wavelength of 405 nm) and aresonator unit 413 for amplifying the laser light emitted from thesemiconductor laser unit 412.

FIG. 13B schematically illustrates a master oscillator power amplifier(MOPA) 414, which is constituted by a master oscillator and an opticalamplifier, as an example of a semiconductor laser that can be used inthe laser light source unit. In the light source, an optical amplifier415 for further amplifying the emitted laser light is provided in alater stage of the master oscillator 411 illustrated in FIG. 13A. Asemiconductor optical amplifier (SOA) or the like can be preferablyused, for example, as the optical amplifier 415.

FIG. 13C schematically illustrates a light source, which has the MOPA414 and a wavelength conversion unit, as an example of a semiconductorlaser that can be used in the laser light source unit. In the lightsource, a wavelength conversion unit 416 for converting a wavelength oflaser light which has had its intensity amplified is provided in a laterstage of the optical amplifier 415 illustrated in FIG. 13B. An opticalparametric oscillator (OPO) which uses various types of non-linearcrystals or the like can be preferably used, for example, as thewavelength conversion unit 416. In addition, by providing a beam shapecorrection unit 417 which corrects beam shapes of laser light betweenthe MOPA 414 and the wavelength conversion unit 416 as illustrated inFIG. 13D, wavelength conversion efficiency of the wavelength conversionunit 416 can be further improved.

In addition, a wavelength of excitation light emitted from the laserlight source is not particularly limited, and a wavelength suitable forexciting a fluorescent substance included in the measurement subject Smay be appropriately selected. Further, a laser to be used as the lightsource may be a continuous wave (CW) laser or a pulse laser.

The image guide fiber 403 is formed by bundling a plurality of multimodeoptical fiber element wires 421 as schematically illustrated in FIG. 14.Each of the optical fiber element wires 421 is formed of a core 423 thatcan guide light of a zero-order mode but also light of a higher-ordermode, and a cladding 425 provided to cover the core 423. The opticalfiber element wires 421 are disposed to have a hexagonal close-packedstructure as schematically illustrated in FIG. 14 so far as possible. Aseparation distance d between adjacent optical fiber element wires 421may be appropriately set in accordance with image resolution to beobtained, and, for example, a value such as 3.5 μm may be possible. Inaddition, a diameter d′ of the core 423 may also be appropriately set,and, for example, a value such as 3 μm may be possible.

The image guide fiber 403 transmits excitation light from the lightsource optical system 401 incident on one end (e.g., an end A in FIG.12) to the measurement subject S, and transmits an image of themeasurement subject S formed on the other end (e.g., an end B in FIG.12) using fluorescence generated from the measurement subject S to theone end (e.g., the end A in FIG. 12).

The imaging optical system 405 scans the image of the measurementsubject S transmitted to the end (e.g., the end A in FIG. 12) of theimage guide fiber 403 at a scanning pitch that is narrower than a sizeof the core 423 (the core diameter d′ in FIG. 14) of each of theplurality of optical fiber element wires 421. At this time, the imagingoptical system 405 performs imaging such that at least a part of anoptical fiber element wire-corresponding image which corresponds to eachof the optical fiber element wires 421 is included in a plurality ofimages, and generates a plurality of pieces of image data of themeasurement subject S.

The imaging optical system 405 is constituted by various detectors whichdetect an image (i.e., a fluorescence signal corresponding to generatedfluorescence) of the measurement subject S transmitted from the imageguide fiber 403, and optical elements such as various lenses, variousmirrors, various filters, and the like which guide the image(fluorescence signal) of the measurement subject S toward the detectors.

A detailed disposition of the various optical elements of the imagingoptical system 405 is not particularly limited, and a disposition of aknown optical system can be employed.

Known detectors can also be used as the detectors provided in theimaging optical system 405 as long as they can convert informationregarding intensity of fluorescence into an electric signal. Variousdetectors such as a charge-coupled device (CCD), a photomultiplier tube(PMT), and the like can be exemplified, for example, as the detector.

In addition, a mechanism for scanning the end faces of the image guidefiber 403 as will be described in detail below is provided in at leastany one of the light source optical system 401 and the imaging opticalsystem 405 shown in FIG. 12. Such a scanning mechanism is notparticularly limited, and, for example, a known mechanism such as agalvano mirror can be used.

The configuration of the imaging unit 40 according to the presentembodiment has been described above in detail with reference to FIGS. 12to 14.

[Scanning Method of Image Guide Fiber]

Next, a scanning method of an end face of the image guide fiber 403 bythe imaging optical system 405 will be described with reference to FIGS.15A to 16B.

First, mode excitation when light converged on an end face of amultimode optical waveguide used in an image guide will be describedwith reference to FIGS. 15A to 15D.

In the multimode optical waveguide (s multimode optical fiber), evenwhen an optical fiber that guides waves in a zero-order mode asillustrated in FIG. 15A and a first-order mode (a higher-order mode) asillustrated in FIG. 15B is used, the modes have orthogonality. Thus, aslong as perturbation caused from the outside, for example, like whenthere is a flaw on the optical waveguide, is not exerted, propagationoccurs on the optical waveguide (optical fiber) in the zero-order modewithout change in the case of the zero-order mode, and propagation alsooccurs on the optical waveguide in the first-order mode without changein the case of the first-order mode.

Next, states of coupling of waveguide modes inside a fiber when lightfrom the outside is coupled on an end face of the fiber (when lightwhich passes through a lens converges on the end face of the opticalfiber) will be described with reference to FIGS. 15C and 15D. Aconvolution integral of an electric field strength distribution ofconvergence spots and an electric field strength distribution ofwaveguide modes are used to decide which waveguide mode will be excitedin the inside of the fiber.

As illustrated in FIG. 15C, when a convergence spot coincides with thecenter of the core and a size of the spot approximates a size of thecore, an electric field distribution of the convergence spotsubstantially matches an electric field distribution of the zero-ordermode illustrated in FIG. 15A, and thus the zero-order mode is excited inthe inside of the fiber. In addition, since the electric field strengthdistribution illustrated in FIG. 15B has a different sign at the center,the electric field strength distribution illustrated in FIG. 15C and theelectric field strength distribution of the first-order mode illustratedin FIG. 15B have an integrated value that is substantially zero, andthus the first-order mode is not excited. On the other hand, when theconvergence spot is shifted from the center of the core as illustratedin FIG. 15D, a convolution integral of the electric field strengthdistribution illustrated in FIG. 15D and the first-order modeillustrated in FIG. 15B are not zero, thus the first-order mode isexcited at a certain ratio, and a ratio at which the zero-order mode isexcited decreases accordingly. Thus, the zero-order mode is excited onthe incident end face when the zero-order mode is dominantly excited onthe incident end face as illustrated in FIG. 15C, and since thewaveguide mode of the optical waveguide has orthogonality, light isguided toward the emission end inside the optical fiber without changein the zero-order mode.

The present inventor has ascertained that, if a single mode (thezero-order mode) is excited on an incident end face of an image guidefiber, a case in which light probably reaches the sample-side end facein a single mode exists, and thus the present inventor has conceived ascanning method for an end face of the image guide fiber 403 asschematically illustrated in FIGS. 16A and 16B.

In other words, the imaging unit 40, according to the present embodimentscans an end face of the image guide fiber 403 (e.g., the end face ofthe end A in FIG. 12) at a scanning pitch that is narrower than a sizeof a core of each of a plurality of optical fiber element wires.

For example, FIG. 16A illustrates an example in which scanning isperformed at a pitch p that is narrower than the size d′ of a core ofeach optical fiber element wire 421 in a direction parallel to ascanning direction of an end face of the image guide fiber 403. In thiscase, a position of an axis expressing the scanning direction (ascanning axis) is set in advance in accordance with a diameter of theimage guide fiber 403 to be used and the core diameter d′ of eachoptical fiber element wire 421, and the end face of the image guidefiber 403 is captured at positions of black circles in FIG. 16A undercontrol of the arithmetic processing unit 50. At this time, althoughimaging intervals in the direction parallel to the scanning directionare controlled by the arithmetic processing unit 50 on the basis of thescanning pitch p, imaging intervals in a direction orthogonal to thescanning direction are controlled on the basis of an interval d of theadjacent optical fiber element wires 421 of the image guide fiber 403.In the scanning method shown in FIG. 16A, the entire measurement subjectS is captured one time under the above-described control. In otherwords, in the scanning method shown in FIG. 16A, a frequency of imagedata generated through imaging (a data restoration frequency) is higherthan the number of optical fiber element wires 421.

In addition, for example, FIG. 16B illustrates an example in whichscanning is performed at the pitch p that is narrower than the size d′of the core of each of the optical fiber element wires 421 in thedirection orthogonal to the scanning direction of an end face of theimage guide fiber 403. In this case, a position of an axis expressingthe scanning direction (a scanning axis) is set in advance in accordancewith the diameter of the image guide fiber 403 to be used and the corediameter d′ of each of the optical fiber element wires 421, and the endface of the image guide fiber 403 is captured at positions of blackcircles in FIG. 16B under control of the arithmetic processing unit 50.At this time, although the imaging intervals in the direction parallelto the scanning direction are controlled by the arithmetic processingunit 50 on the basis of the interval d of the adjacent optical fiberelement wires 421 of the image guide fiber 403, the imaging intervals inthe direction orthogonal to the scanning direction are controlled on thebasis of the scanning pitch p. In the scanning method shown in FIG. 16B,the entire measurement subject S is captured a plurality of times (e.g.,5 times in the example of FIG. 16B) under the above-described control.In other words, in the scanning method shown in FIG. 16B, a frequency ofdata restoration in one scanning corresponds to the number of opticalfiber element wires 421, and reference positions (scanning startpositions) in each scanning process change at the pitch p that isnarrower than a disposition pitch of the optical fiber element wires421.

Note that “imaging” in the above description means forming an image atthe position of the black circles in FIGS. 16A and 16B with excitationlight guided by the light source optical system 401, and capturing animage (a fluorescence image) transmitted to an end face of the imageguide fiber 403 at the positions of the black circles.

As the above-described scanning method is realized, at least a part ofareas corresponding to the optical fiber element wires 421 (which willalso be referred to as an “optical fiber element wires-correspondingarea” below) is captured and included in a plurality of images.

By scanning the end face of the image guide fiber 403 as illustrated inFIGS. 16A and 16B, a plurality of pieces of image data generated throughimaging processes include image data of a case in which excitation lightis radiated to all cores. In this situation, a fundamental wave (thezero-order mode) is excited at the core of the image guide fiber 403,and thus there also is a case in which the fundamental wave probablyreaches a sample-side end face.

Note that the imaging unit 40 may perform an imaging process only onetime at each of the imaging positions illustrated in FIGS. 16A and 16B,and may perform the imaging process a plurality of times at each of theimaging positions to increase a probability.

In addition, it is a matter of course that the imaging unit 40 accordingto the present embodiment may employ a scanning method obtained bycombining the methods of FIG. 16A and FIG. 16B (i.e., the scanningmethods at the scanning pitch p in the scanning direction and thedirection orthogonal to the scanning direction).

A specific size of the scanning pitch p shown in FIGS. 16A and 16B maybe appropriately set in accordance with the core diameter d′ of theoptical fiber element wire 421, and a size of about 1/10 of the corediameter d′ or smaller is preferable. As a result of a review of thepresent inventor, by setting the scanning pitch p to be 1/10 of the corediameter d′ or smaller, (e.g., when the core diameter d′ is 3 μm, thescanning pitch p is set to be 0.3 μm or smaller), it is possible toobtain luminance of 86% or more of the maximum luminance obtainedthrough excitation by the optical fiber element wires 421. In addition,by setting the scanning pitch p to be 1/12 of the core diameter of d′ orsmaller, it is possible to obtain luminance of 90% or more of themaximum luminance obtained through excitation by the optical fiberelement wires 421.

Note that the scanning directions and imaging positions shown in FIGS.16A and 16B and the number of scanning operations shown in FIG. 16B aremerely examples, and the present invention is not limited to theexamples shown in FIGS. 16A and 16B.

The examples of the scanning methods for an end face of the image guidefiber 403 according to the present embodiment have been described abovein detail with reference to FIGS. 16A and 16B.

[Specific Example of Imaging Unit 40]

A specific example of the imaging unit 40 according to the presentembodiment will be briefly described with reference to FIGS. 17 to 18B.

As illustrated in FIG. 17, for example, excitation light computed fromone of the light sources illustrated in FIGS. 13A to 13D is guidedtoward an XY galvano mirror XY-gal via a lens L and a mirror M, and thegalvano mirror controls an image formation position with respect to theimage guide fiber 403. Excitation light that has gone through thegalvano mirror goes through a relay lens RL, the mirror M, and adichroic mirror DM, and is guided to an object lens Obj. Here, one endof the above-described image guide fiber 403 is arranged at a focalposition f of the object lens Obj.

It is preferable to provide an end unit 431 to be described below at theother end of the image guide fiber 403. Accordingly, fluorescence imagesof the measurement subject S having different positions in a thicknessdirection can be easily obtained. Fluorescence is generated from themeasurement subject S by the excitation light that has passed throughthe image guide fiber 403 and the end unit 431 and has been guided tothe measurement subject S. The generated fluorescence passes through theend unit 431, the image guide fiber 403, the object lens Obj, thedichroic mirror DM, another lens L, a notch filter F, and the like, andis guided to a photomultiplier serving as a detector or the like.

Here, with respect to the imaging unit 40 according to the presentembodiment, it is important to acquire fluorescence images whilechanging a position of the measurement subject S in the thicknessdirection (i.e., changing a focal position thereof). Here, as a methodfor changing a focal position, moving a lens 433 installed at a leadingend of the optical fiber element wires 421 in the end unit 431 or movingthe whole end unit 431 to move a coupling face of the lens 433 asillustrated in FIG. 18A is considered. Accordingly, it is possible toacquire fluorescence images which have different observation positionsin the depth direction.

In the method of moving the lens 433, however, it is necessary toprovide a driving mechanism (not illustrated) in the end unit 431, butit is not easy to make a driving mechanism having a narrower diameter,there are cases in which narrowing a diameter of the end unit 431 ischallenging. In addition, in the method of moving the whole end unit431, it is not easy to move the end unit 431 at predetermined intervals,and thus there is a possibility of an error being made in computation ofa scattering coefficient as described in the first embodiment.

Thus, in the imaging unit 40 according to the present embodiment, it ispreferable to provide an optical component 435 for a hologram that cansimultaneously acquire fluorescence from different thickness positionsof the measurement subject S between the optical fiber element wires 421and the lens 433 as illustrated in FIG. 18B. A spatial light modulator(SLM) disclosed in Non-Patent Literatures 2 and 3, a quadraticallydistorted grating disclosed in Non-Patent Literature 4, or the like canbe exemplified, for example, as the optical component 435. By providingthe optical component 435, fluorescence from different thicknesspositions can be acquired in a short period of time such that an imagingplane having different depths is realized without providing a drivingmechanism.

Note that, when the optical component 435 illustrated in FIG. 18B isused, an amount of information regarding the fluorescence at thethickness positions is smaller than the case in which the end unit 431is used as illustrated in FIG. 18A. Thus, when the end unit 431 is usedas illustrated in FIG. 18B, it is preferable to increase an amount ofinformation regarding the fluorescence at the thickness positions byusing the image guide fiber 403 that has as many pixels as possible orby rotating the image guide fiber 403 and then reacquiring the images.Note that, when the image guide fiber 403 is rotated, a direction inwhich the thickness positions are different is also rotated inaccordance with the rotation of the image guide fiber 403 withoutneeding to provide a driving mechanism in the end unit 431.

[Overall Configuration of Arithmetic Processing Unit 50]

Next, a configuration of the arithmetic processing unit 50 included inthe image acquisition system 30 according to the present embodiment willbe described in detail with reference to FIGS. 19 to 23.

FIG. 19 is a block diagram schematically showing a configuration of thearithmetic processing unit included in the image acquisition systemaccording to the present embodiment. FIG. 20 is a block diagram showingan example of a configuration of a data processing unit according to thepresent embodiment. FIGS. 21 to 23 are illustrative diagrams fordescribing a representative pixel value selection process of thearithmetic processing unit according to the present embodiment.

The arithmetic processing unit 50 according to the present embodiment isa unit which controls operations of the imaging unit 40 and performs apredetermined arithmetic process on image data generated by the imagingunit 40, and thereby generates a plurality of fluorescence images eachcorresponding to thickness positions of the measurement subject S.

The arithmetic processing unit 50 mainly has an imaging unit controlunit 501, a data acquisition unit 503, a data processing unit 505, adisplay control unit 507, and a storage unit 509 as schematicallyillustrated in FIG. 19.

The imaging unit control unit 501 is realized by, for example, a CPU, aROM, a RAM, a communication device, and the like. The imaging unitcontrol unit 501 transmits and receives various control signals to andfrom the light source, the optical elements, the scanning mechanism, andthe like constituting the imaging unit 40 according to the presentembodiment, and thereby generally manages various operations of theimaging unit 40. Accordingly, the light source of the imaging unit 40emits excitation light at a predetermined timing, or an end face of theimage guide fiber 403 is scanned on the basis of the above-describedscanning methods. In addition, the light source, the optical elements,the scanning mechanism, and the like constituting the imaging unit 40may also be able to perform various kinds of control while cooperatingwith each other via the imaging unit control unit 501. In addition,various kinds of control information to be used when the imaging unitcontrol unit 501 controls the imaging unit 40 (e.g., informationregarding an imaging position and the like) are output to the dataacquisition unit 503, the data processing unit 505, and the like whennecessary and are appropriately used in various processes performed inthe processing units.

Furthermore, the imaging unit control unit 501 according to the presentembodiment may control power of excitation light radiated toward themeasurement subject S using information regarding a scatteringcoefficient of the measurement subject S computed by the data processingunit 505 to be described below.

The data acquisition unit 503 is realized by, for example, a CPU, a ROM,a RAM, a communication device, and the like. The data acquisition unit503 acquires a plurality of pieces of image data generated by theimaging unit 40 on the basis of the above-described scanning methodsfrom the imaging unit 40. The plurality of pieces of image data acquiredby the data acquisition unit 503 are transmitted to the data processingunit 505 to be described below. In addition, the data acquisition unit503 may associate the acquired plurality of pieces of image data withtime information regarding a date, time and the like at which the imagedata was acquired, and may store the associated data in the storage unit509 to be described below as hi story information.

The data processing unit 505 is realized by, for example, a CPU, a ROM,a RAM, and the like. The data processing unit 505 generates fluorescenceimages of the measurement subject S by performing various data processeson the plurality of images captured by the imaging unit 40 transmittedfrom the data acquisition unit 503, and specifies a position of asurface and the scattering coefficient of the measurement subject Sfurther using the obtained fluorescence images. When the above-describedvarious kinds of information are computed, the data processing unit 505outputs the generated fluorescence images and the computed informationregarding image information to the display control unit 507.Accordingly, the fluorescence images of the measurement subject S andthe image information regarding the fluorescence images are output to adisplay unit (not illustrated) of the arithmetic processing unit 50 orany of a variety of computers and the like that can communicate with thearithmetic processing unit 50. In addition, the data processing unit 505may output the obtained fluorescence images and image information tovarious recording media, various computers, and the like, or may outputthe obtained fluorescence images and image information to a paper mediumor the like using an output device such as a printer. In addition, thedata processing unit 505 may associate the fluorescence images of themeasurement subject S and the image information regarding thefluorescence images with time information regarding a date, time, andthe like at which the information was computed and store the associateddata in the storage unit 509 as history information.

Note that a detailed configuration of the data processing unit 505 willbe described below.

The display control unit 507 is realized by, for example, a CPU, a ROM,a RAM, an output device, and the like. The display control unit 507performs display control to display the fluorescence images of themeasurement subject S, the position of the surface of the measurementsubject S, and various processing results including informationregarding the scattering coefficient and the like of the measurementsubject S transmitted from the data processing unit 505 on an outputdevice such as a display provided in the arithmetic processing unit 50,an output device provided outside of the arithmetic processing unit 50,or the like. Accordingly, a user of the image acquisition system 30 canimmediately ascertain various processing results with regard to themeasurement subject S.

The storage unit 509 is realized by, for example, a RAM, a storagedevice, or the like provided in the arithmetic processing unit 50according to the present embodiment. The storage unit 509 appropriatelyrecords various parameters and process developments that need to besaved by the arithmetic processing unit 50 according to the presentembodiment to perform any process or various databases, programs, andthe like. The storage unit 509 enables the imaging unit control unit501, the data acquisition unit 503, the data processing unit 505, thedisplay control unit 507, and the like to freely perform data read andwrite processes.

[Regarding Configuration of Data Processing Unit 505]

Next, a configuration of the data processing unit 505 provided in thearithmetic processing unit 50 according to the present embodiment willbe described in detail with reference to FIGS. 20 to 23.

The data processing unit 505 according to the present embodiment isprovided with a selection unit 511, a captured image re-constructingunit 513, a representative luminance value specifying unit 515, asurface position specifying unit 517, a scattering coefficientcomputation unit 519, and a result output unit 521 as illustrated inFIG. 20.

The selection unit 511 is realized by, for example, a CPU, a ROM, a RAM,and the like. The selection unit 511 selects, for each of a plurality ofpixels constituting an optical fiber element wire-corresponding area, apixel value that is maximum luminance of a plurality of pieces of imagedata transmitted from the data acquisition unit 503 as a representativepixel value of the pixel. A process of selecting a representative pixelvalue executed by the selection unit 511 will be described in detailbelow with reference to FIGS. 21 to 23.

The image guide fiber 403 is formed by bundling a plurality of opticalfiber element wires 421, an optical fiber element wire-correspondingarea, which is an area corresponding to one of the optical fiber elementwires 421, can be virtually defined on the basis of the interval (thedisposition pitch) d between adjacent optical fiber element wires andthe core diameter d′ of the optical fiber element wire 421 asschematically illustrated in FIG. 21. The selection unit 511 accordingto the present embodiment deals with the optical fiber elementwire-corresponding area defined for each of the optical fiber elementwires 421 as a selection target area in the process for selecting arepresentative pixel value as will be described in detail below.

By realizing the scanning methods shown in FIGS. 16A and 16B by theimaging unit 40, an imaging visual field of a detector provided in theimaging optical system 405 moves in a certain selection target area inaccordance with the elapse of time as schematically illustrated in FIG.22.

Here, in a multi-photon excitation process including a two-photonexcitation process, as optical fiber element wires have a lower-orderoptical waveguide mode, luminance of generated fluorescence increases.In addition, as an imaging target area of an imaging subject at thattime point is positioned closer to the center of a core, luminance ofgenerated fluorescence increases. Thus, by acquiring images a pluralityof times and selecting a highest fluorescence value, it is possible toselectively acquire information of the fluorescence generated throughthe multi-photon excitation process using excitation light that hasreached the sample-side end face in the zero-order mode. Here, thehighest fluorescence value is considered to be given in image datacaptured at a position corresponding to a time T=t_(MAX) in FIG. 22.

Thus, the selection unit 511 refers to the luminance value at positionsof a plurality of pieces of image data including a pixel to be noted foreach of pixels constituting the selection target area and specifies thehighest luminance value. Then, the selection unit 511 uses the specifiedhighest luminance value as a representative pixel value of the pixel tobe noted. As schematically illustrated in FIG. 23, when, for example,there are N pieces of image data 1 to N for pixels to be noted, the Npieces of image data are searched transversally, and image data thatgives a highest luminance value L_(MAX) is used as image data of thepixels to be noted. In the case of FIG. 23, image data k is used asimage data that gives a representative pixel value of the pixels to benoted.

The selection unit 511 executes the process for selecting arepresentative pixel value described above for all selection targetareas (i.e., all optical fiber element wire-corresponding areas).

Here, the image data that gives the maximum luminance value isconsidered to be superimposed with noise accompanying the luminancevalue, and thus the selection unit 511 may select image data that givesa luminance value approximate to the highest luminance value instead ofimage data giving the highest luminance value.

Note that, although the case in which the selection unit 511transversely searches the plurality of pieces of generated image dataand selects image data that gives the highest luminance value has beendescribed in the above description, a specific highest luminance valuecan also be specified using the following method. That is, the selectionunit 511 may specify the highest luminance value by comparing data ofneighboring pixels and performing a filtering process of selecting amaximum luminance value of an optical fiber element wire-correspondingarea. As such a filtering process, for example, an ordinary filteringfor an area of 10 pixels×10 pixels or the like can be exemplified. Usingthe filtering process, the highest luminance value of pixels to be notedcan be more quickly and easily searched for.

When a representative pixel value is selected using the above-describedmethod, the selection unit 511 outputs information regarding theselected representative pixel value to the captured imagere-constructing unit 513.

The captured image re-constructing unit 513 is realized by, for example,a CPU, a ROM, a RAM, and the like. The captured image re-constructingunit 513 re-constructs captured images of the imaging subject S usingthe selected representative pixel value. Accordingly, a fluorescenceimage of the measurement subject S expressing a state of fluorescencegenerated through the multi-photon excitation process can be generated.

Note that the captured image re-constructing unit 513 may cause a blurfilter represented by a Gaussian filter to act on the generatedfluorescence image of the measurement subject S. Accordingly, afluorescence image in which the selected representative pixel value ismore smoothly connected can be obtained.

Furthermore, the captured image re-constructing unit 513 may executeknown post-processing other than the above processes on the generatedfluorescence image.

The captured image re-constructing unit 513 outputs the fluorescenceimage generated as described above to the representative luminance valuespecifying unit 515 and the result output unit 521.

The representative luminance value specifying unit 515, the surfaceposition specifying unit 517, and the scattering coefficient computationunit 519 each have similar configurations and exhibit similar effects tothose of the representative luminance value specifying unit 111, thesurface position specifying unit 113, and the scattering coefficientcomputation unit 115 provided in the information processing device 10according to the first embodiment of the present disclosure, and thusdetailed description thereof will be omitted below.

The result output unit 521 is realized by, for example, a CPU, a ROM, aRAM, a communication device, and the like. The result output unit 521outputs the fluorescence image generated by the captured imagere-constructing unit 513, information regarding a position of a surfaceof the measurement subject S specified by the surface positionspecifying unit 517, or information regarding the scattering coefficientR_(S) of the measurement subject S computed by the scatteringcoefficient computation unit 519.

So far, examples of the functions of the arithmetic processing unit 50according to the present embodiment have been introduced. The respectiveconstituent elements may be configured using universal members andcircuits, or may be configured using hardware specialized for thefunctions of the constituent elements. In addition, all of the functionsof the constituent elements may be fulfilled by a CPU and the like.Thus, a configuration to be used can be appropriately changed inaccordance with a technical level of any occasion at which the presentembodiment is implemented.

Note that a computer program for realizing each function of thearithmetic processing unit according to the present embodiment describedabove can be produced and installed in personal computers and the like.In addition, a computer-readable recording medium on which the computerprogram is stored can also be provided. The recording medium is, forexample, a magnetic disk, an optical disc, a magneto-optical disc, aflash memory, or the like. Furthermore, the computer program may bedistributed through, for example, a network, without using a recordingmedium.

(Hardware Configuration)

Next, the hardware configuration of the information processing device 10and the arithmetic processing unit 50 according to the embodiment of thepresent disclosure will be described in detail with reference to FIG.24. FIG. 24 is a block diagram for illustrating the hardwareconfiguration of the information processing device 10 and the arithmeticprocessing unit 50 according to the embodiment of the presentdisclosure.

The information processing device 10 and the arithmetic processing unit50 mainly include a CPU 901, a ROM 903, and a RAM 905. Furthermore, thearithmetic processing device 20 also includes a host bus 907, a bridge909, an external bus 911, an interface 913, an input device 915, anoutput device 917, a storage device 919, a drive 921, a connection port923, and a communication device 925.

The CPU 901 serves as an arithmetic processing device and a controldevice, and controls the overall operation or a part of the operation ofthe information processing device 10 and the arithmetic processing unit50 according to various programs recorded in the ROM 903, the RAM 905,the storage device 919, or a removable recording medium 927. The ROM 903stores programs, operation parameters, and the like used by the CPU 901.The RAM 905 primarily stores programs used in execution of the CPU 901and parameters and the like varying as appropriate during the execution.These are connected with each other via the host bus 907 configured froman internal bus such as a CPU bus or the like.

The host bus 907 is connected to the external bus 911 such as a PCI(Peripheral Component Interconnect/Interface) bus via the bridge 909.

The input device 915 is an operation means operated by a user, such as amouse, a keyboard, a touch panel, buttons, a switch and a lever. Also,the input device 915 may be a remote control means (a so-called remotecontrol) using, for example, infrared light or other radio waves, or maybe an externally connected device 929 such as a mobile phone or a PDAconforming to the operation of the information processing device 10 andthe arithmetic processing unit 50. Furthermore, the input device 915generates an input signal based on, for example, information which isinput by a user with the above operation means, and is configured froman input control circuit for outputting the input signal to the CPU 901.The user can input various data to the information processing device 10and the arithmetic processing unit 50 and can instruct the informationprocessing device 10 and the arithmetic processing unit 50 to performprocessing by operating this input device 915.

The output device 917 is configured from a device capable of visually oraudibly notifying acquired information to a user. Examples of suchdevice include display devices such as a CRT display device, a liquidcrystal display device, a plasma display device, an EL display deviceand lamps, audio output devices such as a speaker and a headphone, aprinter, a mobile phone, a facsimile machine, and the like. For example,the output device 917 outputs a result obtained by various processingsperformed by the information processing device 10 and the arithmeticprocessing unit 50. More specifically, the display device displays, inthe form of texts or images, a result obtained by various processesperformed by the information processing device 10 and the arithmeticprocessing unit 50. On the other hand, the audio output device convertsan audio signal such as reproduced audio data and sound data into ananalog signal, and outputs the analog signal.

The storage device 919 is a device for storing data configured as anexample of a storage unit of the information processing device 10 andthe arithmetic processing unit 50 and is used to store data. The storagedevice 919 is configured from, for example, a magnetic storage devicesuch as a HDD (Hard Disk Drive), a semiconductor storage device, anoptical storage device, or a magneto-optical storage device. Thisstorage device 919 stores programs to be executed by the CPU 901,various data, and sound signal data, image signal data, or the like,obtained externally.

The drive 921 is a reader/writer for recording medium, and is embeddedin the information processing device 10 and the arithmetic processingunit 50 or attached externally thereto. The drive 921 reads informationrecorded in the attached removable recording medium 927 such as amagnetic disk, an optical disk, a magneto-optical disk, or asemiconductor memory, and outputs the read information to the RAM 905.Furthermore, the drive 921 can write in the attached removable recordingmedium 927 such as a magnetic disk, an optical disk, a magneto-opticaldisk, or a semiconductor memory. The removable recording medium 927 is,for example, a DVD medium, an HD-DVD medium, or a Blu-ray (registeredtrademark) medium. The removable recording medium 927 may be aCompactFlash (CF; registered trademark), a flash memory, an SD memorycard (Secure Digital Memory Card), or the like. Alternatively, theremovable recording medium 927 may be, for example, an IC card(Integrated Circuit Card) equipped with a non-contact IC chip or anelectronic appliance.

The connection port 923 is a port for allowing devices to directlyconnect to the information processing device 10 and the arithmeticprocessing unit 50. Examples of the connection port 923 include a USB(Universal Serial Bus) port, an IEEE1394 port, a SCSI (Small ComputerSystem Interface) port, and the like. Other examples of the connectionport 923 include an RS-232C port, an optical audio terminal, an HDMI(High-Definition Multimedia Interface) port, and the like. By theexternally connected device 929 connecting to this connection port 923,the information processing device 10 and the arithmetic processing unit50 directly obtains sound signal data, image signal data, or the like,from the externally connected device 929 and provides sound signal data,image signal data, or the like, to the externally connected device 929.

The communication device 925 is a communication interface configuredfrom, for example, a communication device for connecting to acommunication network 931. The communication device 925 is, for example,a wired or wireless LAN (Local Area Network), Bluetooth (registeredtrademark), a communication card for WUSB (Wireless USB), or the like.Alternatively, the communication device 925 may be a router for opticalcommunication, a router for ADSL (Asymmetric Digital Subscriber Line), amodem for various communications, or the like. This communication device925 can transmit and receive signals and the like in accordance with apredetermined protocol such as TCP/IP on the Internet and with othercommunication devices, for example. The communication network 931connected to the communication device 925 is configured from a networkand the like, which is connected via wire or wirelessly, and may be, forexample, the Internet, a home LAN, infrared communication, radio wavecommunication, satellite communication, or the like.

Heretofore, an example of the hardware configuration capable ofrealizing the functions of the information processing device 10 and thearithmetic processing unit 50 according to the embodiment of the presentdisclosure has been shown. Each of the structural elements describedabove may be configured using a general-purpose material, or may beconfigured from hardware dedicated to the function of each structuralelement. Accordingly, the hardware configuration to be used can bechanged as appropriate according to the technical level at the time ofcarrying out the present embodiment.

The preferred embodiment(s) of the present disclosure has/have beendescribed above with reference to the accompanying drawings, whilst thepresent disclosure is not limited to the above examples. A personskilled in the art may find various alterations and modifications withinthe scope of the appended claims, and it should be understood that theywill naturally come under the technical scope of the present disclosure.

Further, the effects described in this specification are merelyillustrative or exemplified effects, and are not limitative. That is,with or in the place of the above effects, the technology according tothe present disclosure may achieve other effects that are clear to thoseskilled in the art from the description of this specification.

Additionally, the present technology may also be configured as below.

(1)

An information processing device including:

a representative luminance value specifying unit configured to, whenluminance values constituting a plurality of fluorescence images of ameasurement subject captured while a position of the measurement subjectin a thickness direction is changed are sequentially rearranged from ahighest luminance value on the basis of the fluorescence images for eachof the fluorescence images corresponding to respective thicknesspositions, extract a luminance value ranked at a predetermined positionfrom the highest luminance value and set the extracted luminance valueas a representative luminance value of the fluorescence image at thethickness position to be noted; and

a surface position specifying unit configured to use the representativeluminance value for each of the fluorescence images and set thethickness position corresponding to the fluorescence image that givesthe maximum representative luminance value as a position correspondingto a surface of the measurement subject.

(2)

The information processing device according to (1), wherein thepredetermined ranked position is a position included in a range of a top0.5% to 5% of the number of all pixels constituting one fluorescenceimage with reference to the highest luminance value.

(3)

The information processing device according to (1) or (2), furtherincluding:

a scattering coefficient computation unit configured to compute ascattering coefficient of the measurement subject from a degree ofchange of the representative luminance value in the thickness directionusing the representative luminance value for each of the fluorescenceimages.

(4)

The information processing device according to (3), wherein thescattering coefficient computation unit computes the scatteringcoefficient using the representative luminance values at three thicknesspositions having equal intervals on the basis of the representativeluminance values of the fluorescence images corresponding to deeperparts than the position corresponding to the surface of the measurementsubject.

(5)

The information processing device according to (4),

wherein the fluorescence image is obtained by capturing fluorescencegenerated when the measurement subject is excited with N (N is aninteger that is greater than or equal to 1) photons, and

when the representative luminance value at a thickness position x_(i)(i=1, 2, and 3) is denoted by A_(i) (i=1, 2, and 3), and an intervalbetween the two adjacent thickness positions is denoted by dx, thescattering coefficient computation unit computes a scatteringcoefficient R on the basis of the following formula 1.

(6)

An image acquisition system including:

an imaging unit configured to generate a plurality of pieces of imagedata for fluorescence generated from a measurement subject by radiatingexcitation light toward the measurement subject and imaging thefluorescence of the measurement subject while changing a position of themeasurement subject in a thickness direction; and

an arithmetic processing unit configured to generate a plurality offluorescence images corresponding to respective thickness positions bycontrolling the imaging unit and performing data processing on each ofthe plurality of pieces of image data generated by the imaging unit,

wherein the imaging unit includes

a light source optical system configured to guide excitation light forexciting the measurement subject with two or more photons to generatefluorescence toward the measurement subject,

an image guide fiber which is formed by bundling a plurality ofmultimode optical fiber element wires and is configured to transmit theexcitation light incident on one end to the measurement subject from thelight source optical system and transmit an image of the measurementsubject formed on the other end using the fluorescence generated fromthe measurement subject to the one end, and

an imaging optical system configured to scan the image of themeasurement subject transmitted to the one end of the image guide fiberat a scanning pitch that is narrower than a size of a core of each ofthe plurality of optical fiber element wires to perform imaging suchthat at least a part of an optical fiber element wire-corresponding areawhich corresponds to each of the optical fiber element wires is includedin a plurality of images, and generate a plurality of pieces of imagedata of the measurement subject, and

the arithmetic processing unit includes

a selection unit configured to select, for each of a plurality of pixelsconstituting the optical fiber element wire-corresponding area, a pixelvalue that has maximum luminance among the plurality of pieces of imagedata as a representative pixel value of the pixel,

a captured image re-constructing unit configured to re-construct thecaptured image of the measurement subject using the selectedrepresentative pixel value and generate the fluorescence image,

a representative luminance value specifying unit configured to, whenluminance values constituting the plurality of fluorescence imagescaptured while the position of the measurement subject in the thicknessdirection is changed are sequentially rearranged from a highestluminance value on the basis of the fluorescence images for each of thefluorescence images corresponding to respective thickness positions,extract a luminance value ranked at a predetermined position from thehighest luminance value and set the extracted luminance value as arepresentative luminance value of the fluorescence image at thethickness position to be noted, and

a surface position specifying unit configured to use the representativeluminance value for each of the fluorescence images and set thethickness position corresponding to the fluorescence image that givesthe maximum representative luminance value as a position correspondingto a surface of the measurement subject.

(7)

The image acquisition system according to (6), wherein an opticalcomponent for a hologram which can simultaneously acquire fluorescencefrom different thickness positions of the measurement subject isprovided at a measurement subject-side end of the image guide fiber.

(8)

An information processing method including:

extracting, when luminance values constituting a plurality offluorescence images of a measurement subject captured while a positionof the measurement subject in a thickness direction is changed aresequentially rearranged from a highest luminance value on the basis ofthe fluorescence images for each of the fluorescence imagescorresponding to respective thickness positions, a luminance valueranked at a predetermined position from the highest luminance value andsetting the extracted luminance value as a representative luminancevalue of the fluorescence image to be noted; and

using the representative luminance value for each of the fluorescenceimages and setting the thickness position corresponding to thefluorescence image that gives the maximum representative luminance valueas a position corresponding to a surface of the measurement subject.

(9)

An image information acquisition method including:

guiding excitation light for exciting a measurement subject with two ormore photons to generate fluorescence toward the measurement subject;

transmitting the excitation light incident on one end of an image guidefiber which is formed by bundling a plurality of multimode optical fiberelement wires toward the measurement subject using the image guidefiber, and transmitting an image of the measurement subject formed onthe other end using the fluorescence generated from the measurementsubject to the one end while changing a position of the measurementsubject in a thickness direction;

scanning the image of the measurement subject transmitted to the one endof the image guide fiber at a scanning pitch that is narrower than asize of a core of each of the plurality of optical fiber element wiresto perform imaging such that at least a part of an optical fiber elementwire-corresponding area which corresponds to each of the optical fiberelement wires is included in a plurality of images, and generating aplurality of pieces of image data of the measurement subject;

selecting, for each of a plurality of pixels constituting the opticalfiber element wire-corresponding area, a pixel value that has maximumluminance among the plurality of pieces of image data as arepresentative pixel value of the pixel;

re-constructing the captured image of the measurement subject using theselected representative pixel value and generating a fluorescence image;

extracting, when luminance values constituting a plurality offluorescence images captured while the position of the measurementsubject in the thickness direction is changed are sequentiallyrearranged from a highest luminance value on the basis of thefluorescence images for each of the fluorescence images corresponding torespective thickness positions, a luminance value ranked at apredetermined position from the highest luminance value and setting theextracted luminance value as a representative luminance value of thefluorescence image at the thickness position to be noted; and

using the representative luminance value for each of the fluorescenceimages and setting the thickness position corresponding to thefluorescence image that gives the maximum representative luminance valueas a position corresponding to a surface of the measurement subject.

(10)

A program causing a computer to realize:

a representative luminance value specifying function of extracting, whenluminance values constituting a plurality of fluorescence images of ameasurement subject captured while a position of the measurement subjectin a thickness direction is changed are sequentially rearranged from ahighest luminance value on the basis of the fluorescence images for eachof the fluorescence images corresponding to thickness positions, aluminance value ranked at a predetermined position from the highestluminance value and setting the extracted luminance value as arepresentative luminance value of the fluorescence image to be noted;and

a surface position specifying function of using the representativeluminance value for each of the fluorescence images and setting thethickness position corresponding to the fluorescence image that givesthe maximum representative luminance value as a position correspondingto a surface of the measurement subject.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 3} \right\rbrack & \; \\{S = {{- \frac{1}{N \cdot {dx}}} \cdot {\ln\left( \frac{A_{1} - A_{2}}{A_{2} - A_{3}} \right)}}} & \left( {{Formula}\mspace{14mu} 1} \right)\end{matrix}$

REFERENCE SIGNS LIST

-   10 information processing device-   20, 40 imaging unit-   30 image acquisition system-   50 arithmetic processing unit-   101, 503 data acquisition unit-   103 image information computation unit-   105, 507 display control unit-   107, 509 storage unit-   111, 515 representative luminance value specifying unit-   113, 517 surface position specifying unit-   115, 519 scattering coefficient computation unit-   117, 521 result output unit-   401 light source optical system-   403 image guide fiber-   405 imaging optical system-   421 optical fiber element wire-   423 core-   425 cladding-   501 imaging unit control unit-   505 data processing unit-   511 selection unit-   513 captured image re-constructing unit

The invention claimed is:
 1. An information processing device,comprising: a representative luminance value specifying unit configuredto, when luminance values constituting a plurality of fluorescenceimages of a measurement subject captured while a position of themeasurement subject in a thickness direction is changed are sequentiallyrearranged from a highest luminance value based on the plurality offluorescence images corresponding to respective thickness positions,extract a luminance value ranked at a predetermined position from thehighest luminance value and set the extracted luminance value as arepresentative luminance value of a fluorescence image at a thicknessposition to be noted; and a surface position specifying unit configuredto use the representative luminance value for each of the plurality offluorescence images and set the thickness position corresponding to thefluorescence image that gives a maximum representative luminance valueas a position corresponding to a surface of the measurement subject. 2.The information processing device according to claim 1, wherein thepredetermined position is a position included in a range of a top 0.5%to 5% of a number of all pixels constituting one fluorescence image withreference to the highest luminance value.
 3. The information processingdevice according to claim 1, further comprising: a scatteringcoefficient computation unit configured to compute a scatteringcoefficient of the measurement subject from a degree of change of therepresentative luminance value in the thickness direction using therepresentative luminance value for each of the plurality of fluorescenceimages.
 4. The information processing device according to claim 3,wherein the scattering coefficient computation unit computes thescattering coefficient using representative luminance values at threethickness positions having equal intervals based on the representativeluminance values of fluorescence images corresponding to deeper partsthan the position corresponding to the surface of the measurementsubject.
 5. The information processing device according to claim 4,wherein the fluorescence image is obtained by capture of fluorescencegenerated when the measurement subject is excited with N (N is aninteger that is greater than or equal to 1) photons, and when therepresentative luminance value at a thickness position x_(i) (i=1, 2,and 3) is denoted by Ai (i=1, 2, and 3), and an interval between twoadjacent thickness positions is denoted by dx, the scatteringcoefficient computation unit computes a scattering coefficient Rs basedon the following formula 1 $\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 1} \right\rbrack & \; \\{R_{s} = {{- \frac{1}{N \cdot {dx}}} \cdot {{\ln\left( \frac{A_{1} - A_{2}}{A_{2} - A_{3}} \right)}.}}} & \left( {{Formula}\mspace{14mu} 1} \right)\end{matrix}$
 6. An image acquisition system, comprising: an imagingunit configured to generate a plurality of pieces of image data forfluorescence generated from a measurement subject by radiation of anexcitation light towards the measurement subject and imaging thefluorescence of the measurement subject while a position of themeasurement subject is changed in a thickness direction; and anarithmetic processing unit configured to generate a plurality offluorescence images corresponding to respective thickness positions bycontrolling the imaging unit and perform data processing on each of theplurality of pieces of image data generated by the imaging unit, whereinthe imaging unit includes: a light source optical system configured toguide the excitation light for excitation of the measurement subjectwith at least two photons to generate fluorescence toward themeasurement subject, an image guide fiber which is formed by bundling aplurality of multimode optical fiber element wires and is configured totransmit the excitation light incident on one end to the measurementsubject from the light source optical system and transmit an image ofthe measurement subject formed on an other end using the fluorescencegenerated from the measurement subject to the one end, and an imagingoptical system configured to scan the image of the measurement subjecttransmitted to the one end of the image guide fiber at a scanning pitchthat is narrower than a size of a core of each of the plurality ofmultimode optical fiber element wires to perform imaging such that atleast a part of an optical fiber element wire-corresponding area whichcorresponds to each of the plurality of multimode optical fiber elementwires is included in a plurality of images, and generate the pluralityof pieces of image data of the measurement subject, and the arithmeticprocessing unit includes: a selection unit configured to select, foreach of a plurality of pixels constituting the optical fiber elementwire-corresponding area, a pixel value that has maximum luminance amongthe plurality of pieces of image data as a representative pixel value ofa pixel, a captured image re-constructing unit configured tore-construct a captured image of the measurement subject using theselected representative pixel value and generate a fluorescence image, arepresentative luminance value specifying unit configured to, whenluminance values constituting the plurality of fluorescence imagescaptured while the position of the measurement subject in the thicknessdirection is changed are sequentially rearranged from a highestluminance value based on the plurality of fluorescence imagescorresponding to respective thickness positions, extract a luminancevalue ranked at a predetermined position from the highest luminancevalue and set the extracted luminance value as a representativeluminance value of the fluorescence image at a thickness position to benoted, and a surface position specifying unit configured to use therepresentative luminance value for each of the plurality of fluorescenceimages and set the thickness position corresponding to the fluorescenceimage that gives a maximum representative luminance value as a positioncorresponding to a surface of the measurement subject.
 7. The imageacquisition system according to claim 6, wherein an optical componentfor a hologram which can simultaneously acquire the fluorescence fromdifferent thickness positions of the measurement subject is provided ata measurement subject-side end of the image guide fiber.
 8. Aninformation processing method, comprising: extracting, when luminancevalues constituting a plurality of fluorescence images of a measurementsubject captured while a position of the measurement subject in athickness direction is changed are sequentially rearranged from ahighest luminance value based on the plurality of fluorescence imagescorresponding to respective thickness positions, a luminance valueranked at a predetermined position from the highest luminance value andsetting the extracted luminance value as a representative luminancevalue of a fluorescence image to be noted; and using the representativeluminance value for each of the plurality of fluorescence images andsetting a thickness position corresponding to the fluorescence imagethat gives a maximum representative luminance value as a positioncorresponding to a surface of the measurement subject.
 9. An imageinformation acquisition method, comprising: guiding an excitation lightfor exciting a measurement subject with at least two photons to generatefluorescence toward the measurement subject; transmitting the excitationlight incident on one end of an image guide fiber which is formed bybundling a plurality of multimode optical fiber element wires toward themeasurement subject using the image guide fiber, and transmitting animage of the measurement subject formed on an other end using thefluorescence generated from the measurement subject to the one end whilechanging a position of the measurement subject in a thickness direction;scanning the image of the measurement subject transmitted to the one endof the image guide fiber at a scanning pitch that is narrower than asize of a core of each of the plurality of multimode optical fiberelement wires to perform imaging such that at least a part of an opticalfiber element wire-corresponding area which corresponds to each of theplurality of multimode optical fiber element wires is included in aplurality of images, and generating a plurality of pieces of image dataof the measurement subject; selecting, for each of a plurality of pixelsconstituting the optical fiber element wire- corresponding area, a pixelvalue that has maximum luminance among the plurality of pieces of imagedata as a representative pixel value of a pixel; re-constructing acaptured image of the measurement subject using the selectedrepresentative pixel value and generating a fluorescence image;extracting, when luminance values constituting a plurality offluorescence images captured while the position of the measurementsubject in the thickness direction is changed are sequentiallyrearranged from a highest luminance value based on the plurality offluorescence images corresponding to respective thickness positions, aluminance value ranked at a predetermined position from the highestluminance value and setting the extracted luminance value as arepresentative luminance value of the fluorescence image at a thicknessposition to be noted; and using the representative luminance value foreach of the plurality of fluorescence images and setting the thicknessposition corresponding to the fluorescence image that gives a maximumrepresentative luminance value as a position corresponding to a surfaceof the measurement subject.